Papers with learning problem
Align-to-Distill: Trainable Attention Alignment for Knowledge Distillation in Neural Machine Translation (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing knowledge distillation approaches to NMT often rely on heuristics when deciding which teacher layers to distill from. |
| Approach: | They propose an approach to align student attention heads with their teacher counterparts by heuristics to solve a feature mapping problem. |
| Outcome: | The proposed strategy shows gains of +3.61 and +0.63 BLEU points for WMT-2022 DeDsb and WMT-2014 EnDe compared to baselines. |
Natural Language to Structured Query Generation via Meta-Learning (N18-2)
Copied to clipboard
| Challenge: | Conventional supervised training is a pervasive paradigm for NLP problems . however, examples of the same problem may vary widely . a few-shot meta-learning scenario is used to learn multiple models . |
| Approach: | They propose a learning protocol that treats each example as a unique pseudo-task . they use a few-shot meta-learning scenario to reduce the original learning problem to a single example . |
| Outcome: | The proposed learning protocol achieves 1.1%–5.4% accuracy gains over non-meta-learning counterparts on a WikiSQL dataset. |
PlotCoder: Hierarchical Decoding for Synthesizing Visualization Code in Programmatic Context (2021.acl-long)
Copied to clipboard
| Challenge: | Creating effective visualizations is a challenge given the myriad of parameters that users need to provide. |
| Approach: | They propose to synthesize visualization programs from natural language utterances and programmatic context using PlotCoder. |
| Outcome: | The proposed architecture models both the code context and the input utterance. |
TableGPT: Few-shot Table-to-Text Generation with Table Structure Reconstruction and Content Matching (2020.coling-main)
Copied to clipboard
| Challenge: | Recent studies show that pre-trained language models can produce informative and fluent text with the help of large-scale datasets, but they suffer insufficient learning problem with limited training data. |
| Approach: | They propose to use table transformation module with template to rewrite structured table in natural language as input for GPT-2 and exploit multi-task learning with two auxiliary tasks to preserve table’s structural information. |
| Outcome: | The proposed model outperforms existing systems on most few-shot settings. |
ConTinTin: Continual Learning from Task Instructions (2022.acl-long)
Copied to clipboard
| Challenge: | a new learning paradigm is proposed for NLP, which seeks supervision for solving a target task. |
| Approach: | They propose a new learning paradigm that uses textual instructions to learn new tasks . the main goal of machine learning algorithms lies in seeking supervision for solving a target task. |
| Outcome: | The proposed learning paradigm is based on a stream of more than 60 tasks . it makes full use of task instructions to improve forward-transfer and backward-transference . |
Distantly Supervised Named Entity Recognition using Positive-Unlabeled Learning (P19-1)
Copied to clipboard
| Challenge: | Empirical studies on four public NER datasets demonstrate the effectiveness of our proposed method. |
| Approach: | They propose a method to perform named entity recognition using unlabeled data and named entity dictionaries. |
| Outcome: | The proposed method can estimate task loss as if there is fully labeled data. |
A Simple and Effective Model for Answering Multi-span Questions (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing models for reading comprehension restrict output space to a set of single contiguous spans . multi-span questions are problematic because they require multiple inputs - a task that requires a sequence tagging problem . |
| Approach: | They propose a simple architecture for answering multi-span questions by casting the task as a sequence tagging problem. |
| Outcome: | The proposed model significantly improves performance on span extraction questions from DROP and Quoref by 9.9 and 5.5 EM points respectively. |
Pre-training for Abstractive Document Summarization by Reinstating Source Text (2020.emnlp-main)
Copied to clipboard
| Challenge: | Abstractive document summarization models are often trained on limited supervised data . authors present three objectives for pretraining abstractive summarizing models . |
| Approach: | They propose to pre-train a SEQ2SEQ based abstractive summarization model on unlabeled text. |
| Outcome: | The proposed method improves on two benchmark summarization datasets with 19GB of text . the goal is sentence reordering, next sentence generation and masked document generation . |
Handling Extreme Class Imbalance in Technical Logbook Datasets (2021.acl-long)
Copied to clipboard
| Challenge: | Technical logbooks are a challenging and under-explored text type in automated event identification. |
| Approach: | They propose a feedback strategy that resamples the training data based on its error in the prediction process. |
| Outcome: | The proposed approach provides the best results for four different neural network models trained across a suite of technical logbook datasets from distinct technical domains. |
Exploiting Careful Design of SVM Solution for Aspect-term Sentiment Analysis (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Aspect-term sentiment analysis (ATSA) identifies fine-grained sentiments towards specific aspects of text. |
| Approach: | They propose a pipeline to predict fine-grained sentiments for specific aspects of text . it decomposes the learning problem into multiple view subproblems and dynamically selects and constructs features with reinforcement learning. |
| Outcome: | The proposed pipeline surpasses SVM-based methods in predictive accuracy while maintaining a faster inference speed and significantly reducing the number of model parameters. |
Scaling LLM Inference Efficiently with Optimized Sample Compute Allocation (2025.naacl-long)
Copied to clipboard
| Challenge: | Existing methods to optimize sample allocations for large language models fail to account for the optimal sampling configuration. |
| Approach: | They propose an algorithm that optimizes sample allocation by finding an optimal mix of different inference configurations. |
| Outcome: | The proposed algorithm achieves better accuracy on SWE-Bench with 3x less compute than the default configuration. |
Automatic In-the-wild Dataset Annotation with Deep Generalized Multiple Instance Learning (2020.lrec-1)
Copied to clipboard
| Challenge: | Existing methods to label large datasets that resemble real life situations are prohibitive due to the cost of manual labeling. |
| Approach: | They propose to automate the annotation process by using end-to-end differentiable neural networks to label large datasets that resemble real life conditions. |
| Outcome: | The proposed method can label a large dataset in the wild without human intervention without any cost. |
Learning to Play Like Humans: A Framework for LLM Adaptation in Interactive Fiction Games (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches prioritize task-specific performance metrics over human-like comprehension of narrative context and gameplay logic. |
| Approach: | They propose a framework that guides Large Language Models to learn and play IF games systematically. |
| Outcome: | The proposed framework aligns LLMs-based agents’ behavior with narrative intent and commonsense constraints to deliver more interpretable, human-like performance. |
Modeling Graph Structure in Transformer for Better AMR-to-Text Generation (D19-1)
Copied to clipboard
| Challenge: | Recent studies on AMR-to-text generation formalize the task as a sequence-tosequence learning problem . previous approaches only consider the relations between directly connected concepts while ignoring the rich structure in AMR graphs. |
| Approach: | They propose a structure-aware self-attention approach to model the relations between indirectly connected concepts in the seq2seq model. |
| Outcome: | The proposed approach outperforms the state-of-the-art on English AMR benchmarks . it significantly outperformed the state of the art on the benchmarks, with 29.66 and 31.82 BLEU scores . |
MedDCR: Learning to Design Agentic Workflows for Medical Coding (2026.findings-acl)
Copied to clipboard
| Challenge: | Medical coding is the process of translating unstructured clinical notes into standardized diagnostic and procedural codes. |
| Approach: | They propose a closed-loop framework that treats workflow design as a learning problem. |
| Outcome: | The proposed framework outperforms state-of-the-art workflows on benchmark datasets and produces interpretable, adaptable workflows that better reflect real coding practice. |
Constraint-based Learning of Phonological Processes (D19-1)
Copied to clipboard
| Challenge: | Phonological processes govern the way speech sounds in natural languages change depending on context . a novel approach to learning phonological processes from related utterances is proposed . |
| Approach: | They propose an unsupervised approach to learning phonological processes from related utterances . they encode the problem into Boolean constraints that enable data efficiency and fast inference . |
| Outcome: | The proposed approach achieves high accuracy at interactive speeds on phonology problems and datasets. |
Learning from Children: Improving Image-Caption Pretraining via Curriculum (2023.findings-acl)
Copied to clipboard
| Challenge: | Image-caption pretraining is a difficult problem as it requires multiple concepts (nouns) from captions to be aligned to multiple objects in images. |
| Approach: | They propose a curriculum learning framework that uses images to align multiple concepts to multiple objects in an image. |
| Outcome: | The proposed learning framework improves over pretraining from scratch, using a pretrained image or/and text encoder, low data regime etc. |
EpiCaR: Knowing What You Don’t Know Matters for Better Reasoning in LLMs (2026.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to improving reasoning abilities of large language models incur a significant calibration cost. |
| Approach: | They propose an epistemic learning problem that integrates reasoning and calibration into an iterative supervised training framework. |
| Outcome: | The proposed method achieves Pareto-superiority over standard baselines in accuracy and calibration. |